On-line oil and foreign matter detection system and method employing hyperspectral imaging

ABSTRACT

A method for removing foreign matter from an agricultural product stream of a manufacturing process. The method includes conveying a product stream past an inspection station; scanning a region of the agricultural product stream as it passes the inspection station using at least one light source of a single or different wavelengths; generating hyperspectral images from the scanned region; determining a spectral fingerprint for the agricultural product stream from the hyperspectral images; comparing the spectral fingerprint obtained in step (c) to a spectral fingerprint database containing a plurality of fingerprints using a computer processor to determine whether foreign matter is present and, if present, generating a signal in response thereto; and removing a portion of the conveyed product stream in response to the signal. A system for detecting foreign matter within an agricultural product stream is also provided.

RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 14/443,990, filedMay 19, 2015, allowed, which is a National Stage Entry ofPCT/US2013/070812, filed Nov. 19, 2013, which claims priority to U.S.Ser. No. 61/728,123, filed Nov. 19, 2012, the contents of each arehereby incorporated by reference herein in their entireties.

FIELD

Disclosed herein is an on-line system and method for the detection andseparation of unwanted materials and/or foreign matter, usinghyperspectral imaging and analysis.

ENVIRONMENT

Tobacco delivered for processing into filler for cigarettes mayoccasionally contain foreign matter such as pieces of the container inwhich it is shipped and/or stored, bits of string and paper, and otheritems. There remains a need for methods and systems to remove foreign,non-tobacco related materials (NTRM or foreign material).

Another area where it is important to optically inspect and sort amoving stream of product is in the food-processing industry where thereis a need to automatically sort food products by optical inspection ofthe food products to identify food articles having undesirable visualcharacteristics or intermixed foreign material. Examples include fruits,vegetables, baked products, nuts and the like. Other areas requiring asimilar sorting of products or articles includes the sorting ofnaturally occurring products such as meat products, particularly ofquartered or cubed poultry or beef products. In the processing andpackaging of comestible consumer products and products designed toprovide tobacco enjoyment, oils, greases and lubricants may come intocontact with the product being produced.

In the case of products designed to provide tobacco enjoyment, tobaccoleaf may be contacted by machinery during harvesting, curing andtransport to a stemmery. When leaf is provided in strip form at astemmery, and cut or otherwise shredded to the desired size, whileunlikely, oils, greases and lubricants can come into contact with thetobacco. Likewise, lubricants used in operating the various machinesused in the processing of the tobacco can come into contact with thattobacco. The sources of lubricant can vary, such as when a particularpiece of machinery or component of that piece of machinery fails tooperate in an optimum fashion.

Although extremely unlikely, lubricants may come into contact withtobacco due to leakage of lubricants through gaskets or seals, fromsliding mechanisms, from drum systems, from gear boxes, from pumps, fromsealed rolling bearing units, from chains and belts, and the like.Lubricants are used in conditioning cylinders, threshers, separators,redryers, receivers, feeders, conveyors, cutters, blenders, tobaccopresses and other such pieces of equipment that are commonly used intobacco stemmeries and in tobacco primary processing operations. Foreignmatter may sometimes be introduced during harvesting, baling,transporting and bundling operations.

Since lubricants of similar compositions are used throughout the variousstages of tobacco treatment and cigarette manufacture, it is oftendifficult for the cigarette manufacturer to locate the source of aparticular lubricant. As such, the cigarette manufacturer is forced toconduct a time consuming search for the source of the lubricant, inorder to identify and remove the material that may have come in contactwith it.

As in the case of any agricultural product, tobacco may be characterizedby a wide variety of physical, chemical, and/or biological properties,characteristics, features, and behavior, which are associated withvarious aspects relating to agriculture, agronomy, horticulture, botany,environment, geography, climate, and ecology of the tobacco crop andplants thereof from which tobacco leaves are derived, as well as themanner in which the tobacco has been processed, aged or fermented toproduce the sensorial characteristics sought to be achieved by suchfurther processing. Moreover, as those skilled in the art will plainlyrecognize, these characteristics can vary in time throughout furtherprocessing, aging or fermentation.

In the general technique of hyperspectral imaging, one or more objectsin a scene or sample are affected in a way, such as excitation byincident electromagnetic radiation supplied by an external source ofelectromagnetic radiation upon the objects, which causes each object toreflect, scatter and/or emit electromagnetic radiation featuring aspectrum.

Hyperspectral imaging and analysis is a combined spectroscopy andimaging type of analytical method involving the sciences andtechnologies of spectroscopy and imaging. By definition, hyperspectralimaging and analysis is based on a combination of spectroscopy andimaging theories, which are exploitable for analyzing samples ofphysical, chemical, and/or biological matter in a highly unique,specialized, and sophisticated, manner.

Hyperspectral images generated by and collected from a sample of mattermay be processed and analyzed by using automatic pattern recognitionand/or classification type data and information processing and analysis,for identifying, characterizing, and/or classifying, the physical,chemical, and/or biological properties of the hyperspectrally imagedobjects in the sample of matter.

None of the commercially available optical scanning and sorting systemscan detect and reject product that has come in contact with oil orlubricant and/or other NTRM. As such, it would be advantageous if theinspection for oils, greases, lubricants, NTRM and other undesirablematerials on or intermixed with consumer products, such as food, tobaccoand tobacco products could be conducted on-line, that is, in real time,using an optical scanning and sorting system during the productionprocess.

SUMMARY

Disclosed herein is a method for removing foreign matter (including oilor lubricant in product) from an agricultural product stream of amanufacturing process. The method includes conveying a product streampast an inspection station; scanning a region of the agriculturalproduct stream as it passes the inspection station using at least onelight source of a single or different wavelengths; generatinghyperspectral images from the scanned region; determining a spectralfingerprint for the agricultural product stream from the hyperspectralimages; comparing the spectral fingerprint so obtained to a spectralfingerprint database containing a plurality of fingerprints using acomputer processor to determine whether foreign matter is present and,if present, generating a signal in response thereto; and removing aportion of the conveyed product stream in response to the signal.

In some forms, the method includes the step of causing the portion ofthe conveyed product stream to fall under the influence of gravity in acascade.

In some forms, the cascade is a turbulent cascade.

In some forms, the step of removing a portion of the conveyedagricultural product stream in response to the signal further includesdirecting fluid under pressure at the portion of the agriculturalproduct stream.

In some forms, the fluid is a gas.

In some forms, the gas is pressurized air.

In some forms, the agricultural product is tobacco.

In some forms, the agricultural product is tea.

In some forms, the at least one light source is positioned to minimizethe angle of incidence of each beam of light with the agriculturalproduct stream.

In some forms, the at least one light source for providing a beam oflight comprises a light source selected from the group consisting of atungsten light source, a halogen light source, a xenon light source, amercury light source, an ultraviolet light source, and combinationsthereof.

In a further aspect, provided is a system for detecting foreign matter(including oil or lubricant in product) within an agricultural productstream. The system includes a first conveying means for delivering aproduct stream; an inspection station comprising (i) at least one lightsource of a single or different wavelengths for providing a beam oflight to scan a region of the agricultural product stream as it passesthe inspection station, and (ii) a hyperspectral camera system forproviding a three dimensional hyperspectral image cube; a computerprocessor structured and arranged to determine a spectral fingerprintfor the agricultural product stream from the hyperspectral image cubeand to compare the spectral fingerprint obtained to a spectralfingerprint database containing a plurality of fingerprints to determinewhether foreign matter is present and, if present, generating a signalin response thereto.

In some forms, the system includes at least one deflecting systemresponsive to the signals obtained from the computer processor, the atleast one deflecting system directing fluid under pressure at a portionof the product stream when the computer processor determines thatforeign matter is present in the product stream.

In some forms, the fluid so directed is effective to remove the foreignmatter.

In some forms, the system includes a second conveying means locatedbelow and spaced vertically from the first conveying means for furtherconveying the product stream from the first conveying means, wherein theproduct stream is transferred from the first conveying means to thesecond conveying means by falling therebetween under the influence ofgravity in a cascade.

In some forms, the cascade is a turbulent cascade.

In some forms, the first conveying means is an inclined vibratingconveyor.

In some forms, the fluid is a gas.

In some forms, the gas is air.

In some forms, the agricultural product is tobacco.

In some forms, the agricultural product is tea.

In some forms, the at least one light source is positioned to minimizethe angle of incidence of each beam of light with the agriculturalproduct stream.

In some forms, the at least one light source for providing a beam oflight comprises a light source selected from the group consisting of atungsten light source, a halogen light source, a xenon light source, amercury light source, an ultraviolet light source, and combinationsthereof.

In yet another aspect, disclosed herein is a method of creating adatabase for use in identifying foreign material (including oil orlubricant) agricultural product that may be present in a manufacturingprocess for producing an agricultural product. The method utilizeshyperspectral imaging and includes the steps of (a) obtaining a darkimage and a reference image for calibration; (b) analyzing the referenceimage to obtain calibration coefficients; (c) obtaining a hyperspectralimage for an agricultural sample; (d) removing dark values andnormalizing the agricultural sample image; (e) applying calibrationcoefficients to compensate for fluctuations in system operatingconditions; (f) repeating steps (c)-(e) for all agricultural samples;(g) obtaining a hyperspectral image for a foreign material sample; (h)removing dark values and normalizing the agricultural sample image; (i)applying calibration coefficients to compensate for fluctuations insystem operating conditions; (j) repeating steps (g)-(i) for all samplesand (k) storing all hyperspectral sample hypercubes to form thedatabase.

In one form, the computer database is stored in a computer readablemedium.

Certain forms disclosed herein are implemented by performing steps orprocedures, and sub-steps or sub-procedures, in a manner selected fromthe group consisting of manually, semi-automatically, fullyautomatically, and combinations thereof, involving use and operation ofsystem units, system sub-units, devices, assemblies, sub-assemblies,mechanisms, structures, components, and elements, and, peripheralequipment, utilities, accessories, and materials. Moreover, according toactual steps or procedures, sub-steps or sub-procedures, system units,system sub-units, devices, assemblies, sub-assemblies, mechanisms,structures, components, and elements, and, peripheral equipment,utilities, accessories, and materials, used for implementing aparticular form, the steps or procedures, and sub-steps orsub-procedures are performed by using hardware, software, and/or anintegrated combination thereof, and the system units, sub-units,devices, assemblies, sub-assemblies, mechanisms, structures, components,and elements, and peripheral equipment, utilities, accessories, andmaterials, operate by using hardware, software, and/or an integratedcombination thereof.

For example, software used, via an operating system, for implementingcertain forms disclosed herein can include operatively interfaced,integrated, connected, and/or functioning written and/or printed data,in the form of software programs, software routines, softwaresubroutines, software symbolic languages, software code, softwareinstructions or protocols, software algorithms, or a combinationthereof. For example, hardware used for implementing certain formsdisclosed herein can include operatively interfaced, integrated,connected, and/or functioning electrical, electronic and/orelectromechanical system units, sub-units, devices, assemblies,sub-assemblies, mechanisms, structures, components, and elements, and,peripheral equipment, utilities, accessories, and materials, which mayinclude one or more computer chips, integrated circuits, electroniccircuits, electronic sub-circuits, hard-wired electrical circuits, or acombination thereof, involving digital and/or analog operations. Certainforms disclosed herein can be implemented by using an integratedcombination of the just described exemplary software and hardware.

In certain forms disclosed herein, steps or procedures, and sub-steps orsub-procedures can be performed by a data processor, such as a computingplatform, for executing a plurality of instructions. Optionally, thedata processor includes volatile memory for storing instructions and/ordata, and/or includes non-volatile storage, for example, a magnetichard-disk and/or removable media, for storing instructions and/or data.Optionally, certain forms disclosed herein include a network connection.Optionally, certain forms disclosed herein include a display device anda user input device, such as a touch screen device, keyboard and/ormouse.

BRIEF DESCRIPTION OF THE DRAWINGS

The forms disclosed herein are illustrated by way of example, and not byway of limitation, for the case of processing tobacco for use inmanufactured tobacco products in the figures of the accompanyingdrawings and in which like reference numerals refer to similar elementsand in which:

FIG. 1 is a block diagram showing the various stages in the process ofcigarette manufacturing;

FIG. 2 presents a schematic representation of a detection and separationsystem, in accordance herewith;

FIG. 3 presents a method for analyzing data to create a spectrallibrary, in accordance herewith;

FIG. 4 presents a method for analyzing data to create a spectrallibrary, in accordance herewith; and

FIG. 5 presents a graph of intensity as a function wavelength fortobacco and non-tobacco related materials, the graph produced inaccordance herewith.

DETAILED DESCRIPTION

Various aspects will now be described with reference to specific formsselected for purposes of illustration. It will be appreciated by thoseskilled in the art that the spirit and scope of the apparatus, systemand methods disclosed herein are not limited to the selected forms.Moreover, it is to be noted that the figures provided herein are notdrawn to any particular proportion or scale, and that many variationscan be made to the illustrated forms. Reference is now made to FIGS.1-5, wherein like numerals are used to designate like elementsthroughout.

Each of the following terms written in singular grammatical form: “a,”“an,” and “the,” as used herein, may also refer to, and encompass, aplurality of the stated entity or object, unless otherwise specificallydefined or stated herein, or, unless the context clearly dictatesotherwise. For example, the phrases “a device,” “an assembly,” “amechanism,” “a component,” and “an element,” as used herein, may alsorefer to, and encompass, a plurality of devices, a plurality ofassemblies, a plurality of mechanisms, a plurality of components, and aplurality of elements, respectively.

Each of the following terms: “includes,” “including,” “has,” “having,”“comprises,” and “comprising,” and, their linguistic or grammaticalvariants, derivatives, and/or conjugates, as used herein, means“including, but not limited to.”

It is to be understood that the various forms disclosed herein are notlimited in their application to the details of the order or sequence,and number, of steps or procedures, and sub-steps or sub-procedures, ofoperation or implementation of forms of the method or to the details oftype, composition, construction, arrangement, order and number of thesystem, system sub-units, devices, assemblies, sub-assemblies,mechanisms, structures, components, elements, and configurations, and,peripheral equipment, utilities, accessories, and materials of forms ofthe system, set forth in the following illustrative description,accompanying drawings, and examples, unless otherwise specificallystated herein. The apparatus, systems and methods disclosed herein canbe practiced or implemented according to various other alternative formsand in various other alternative ways, as can be appreciated by thoseskilled in the art.

It is also to be understood that all technical and scientific words,terms, and/or phrases, used herein throughout the present disclosurehave either the identical or similar meaning as commonly understood byone of ordinary skill in the art, unless otherwise specifically definedor stated herein. Phraseology, terminology, and, notation, employedherein throughout the present disclosure are for the purpose ofdescription and should not be regarded as limiting.

Moreover, all technical and scientific words, terms, and/or phrases,introduced, defined, described, and/or exemplified, in the abovesections, are equally or similarly applicable in the illustrativedescription, examples and appended claims.

Steps or procedures, sub-steps or sub-procedures, and, equipment andmaterials, system units, system sub-units, devices, assemblies,sub-assemblies, mechanisms, structures, components, elements, andconfigurations, and, peripheral equipment, utilities, accessories, andmaterials, as well as operation and implementation, of exemplary forms,alternative forms, specific configurations, and, additional and optionalaspects, characteristics, or features, thereof, of the methods, and ofthe systems, disclosed herein, are better understood with reference tothe following illustrative description and accompanying drawings.Throughout the following illustrative description and accompanyingdrawings, same reference notation and terminology (i.e., numbers,letters, and/or symbols), refer to same system units, system sub-units,devices, assemblies, sub-assemblies, mechanisms, structures, components,elements, and configurations, and, peripheral equipment, utilities,accessories, and materials, components, elements, and/or parameters.

As a means of illustration, the system will be described for applicationduring tobacco processing, but substantially the same system could beapplied during the processing of other agricultural products. Thedetection and separation system disclosed herein can be used in manyprocesses and for consumer products which are susceptible to thepresence of unwanted materials during the manufacturing process, such asfor example in the growing, collection, processing and/or packaging ofpackaged consumer goods, such as food products, beverages, tipped andnon-tipped cigars, cigarillos, snus and other smokeless tobaccoproducts, smoking articles, electronic cigarettes, distilled products,pharmaceuticals, frozen foods and other comestibles, and the like.Further applications could include clothing, furniture, lumber or anyother manufactured or packaged product wherein an absence of oil isdesired.

Referring now to FIG. 1, a block diagram showing the various stages inthe process of cigarette manufacturing is presented. As shown, tobaccois first harvested at farm 10, which, in the case of tobacco for use incigarette manufacturing or the production of moist smokeless tobacco(MST), will be harvested at least in part by machinery. Tobacco in theform of leaf is baled and received at a receiving station 20 from farm10. Again, the opportunity exists for the tobacco bale to come incontact with lubricated machinery at receiving station 20. The baledtobacco may be transferred to a stemmery 30 wherein large stems areremoved by machines to produce destemmed tobacco. The destemmed tobaccois packed into bales which are then stored for a suitable time period ofup to several years. Destemmed tobacco is then transferred tomanufacturing center 40, wherein various types of tobacco strip may bemachine blended according to a predetermined recipe. The blended tobaccomay be treated by adding various flavorants to provide a cased tobacco,which is cut at 20-40 cuts per inch to provide tobacco “cut filler.”Various other types of tobacco can be added to the cut filler includingpuffed tobacco, reconstituted tobacco, tobacco reclaimed from rejectedcigarettes, and the like, to provide a final product blend. The blendmay be then fed to make/pack machine 50, which includes a continuouscigarette rod making apparatus. The continuous rod is then cut,optionally tipped, and packed, typically through the use of high-speedmachinery.

As may be appreciated from the above description, in tobacco processing,tobacco comes into contact with machinery at many different points inthe overall process, such as machinery used during the growing andharvesting operations on the farm, handling equipment at the receivingstation or auction house, machinery in the stemmery, on conveyors,conditioners, cutters and silos in the primary manufacturing centers,and ultimately on makers, tippers and packers in the make/packmanufacturing centers.

The forms disclosed herein are generally focused on the domainsencompassing the manufacturing or processing of tobacco, blendcomponents or samples, and are specifically focused on the domainsencompassing the automatic monitoring of tobacco processing, performedvia hyperspectral imaging and analysis. However, it should be understoodthat the forms disclosed herein could be applied to other domainsencompassing the manufacturing or processing of tea, fruits, during theproduction of fruit juices, grapes for the production of wines, as wellas a vast array of other agricultural products.

In hyperspectral imaging, a field of view of a sample is scanned andimaged while the sample is exposed to electromagnetic radiation. Duringthe hyperspectral scanning and imaging there is generated and collectedrelatively large numbers of multiple hyperspectral images,one-at-a-time, but, in an extremely fast sequential manner of theobjects emitting electromagnetic radiation at a plurality of wavelengthsand frequencies, where the wavelengths and frequencies are associatedwith different selected portions or bands of an entire hyperspectrumemitted by the objects. A hyperspectral imaging and analysis system canbe operated in an extremely rapid manner for providing exceptionallyhighly resolved spectral and spatial data and information of an imagedsample of matter, with high accuracy and high precision, which arefundamentally unattainable by using standard spectral imaging andanalysis.

In general, when electromagnetic radiation in the form of light, such asthat used during hyperspectral imaging, is incident upon an object, theelectromagnetic radiation is affected by one or more of the physical,chemical, and/or biological species or components making up the object,by any combination of electromagnetic radiation absorption, diffusion,reflection, diffraction, scattering, and/or transmission, mechanisms.Moreover, an object whose composition includes organic chemical speciesor components, ordinarily exhibits some degree of fluorescent and/orphosphorescent properties, when illuminated by some type ofelectromagnetic radiation or light, such as ultra-violet (UV), visible(VIS), or infrared (IR), types of light. The affected electromagneticradiation, in the form of diffused, reflected, diffracted, scattered,and/or transmitted, electromagnetic radiation emitted by the object isdirectly and uniquely related to the physical, chemical, and/orbiological properties of the object, in general, and of the chemicalspecies or components making up the object, in particular, and thereforerepresents a unique spectral fingerprint or signature pattern type ofidentification and characterization of the object.

A typical hyperspectral imaging system consists of an automatedmeasurement system and corresponding analysis software. The automatedmeasurement system includes optics, mechanics, electronics, andperipheral hardware and software, for irradiating, typically using anilluminating source, a scene or sample, followed by measuring andcollecting light emitted, for example, by fluorescence, from objects inthe scene or sample, and for applying calibration techniques best suitedfor extracting desired results from the measurements. Analysis softwareincludes software and mathematical algorithms for analyzing, displaying,and presenting, useful results about the objects in the scene or samplein a meaningful way.

The hyperspectral image of a scene or a sample may be obtained by usingone or more commercially available hyperspectral imaging cameras, suchas those obtainable from Surface Optics Corporation of San Diego,Calif., or others, or custom built according to the user needs.

Hyperspectral imaging can be thought of as a combination of spectroscopyand imaging. In spectroscopy spectrum is collected at a single point.Spectra contain information about the chemical composition and materialproperties of a sample, and consist of a continuum of values thatcorrespond to measurements at different wavelengths of light. Incontrast, traditional cameras collect data of thousands of points. Eachpoint or pixel contains one value (black and white image) or threevalues for a color image, corresponding to colors, red, green, and blue.Hyperspectral cameras combine the spectral resolution of spectroscopyand the spatial resolution of cameras. They create images with thousandsof pixels that contain an array of values corresponding to lightmeasurements at different wavelengths. Or, in other words, the data ateach pixel is a spectrum. Together the pixels and the correspondingspectra create a multi-dimensional image cube. The amount of informationcontained in an image cube is immense, and provides a very detaileddescription of the underlying sample.

Each hyperspectral image is a three dimensional data set of voxels(volume of pixels) in which two dimensions are spatial coordinates orposition, (x, y), in an object and the third dimension is thewavelength, (λ), of the emitted light of the object, such thatcoordinates of each voxel in a spectral image may be represented as (x,y, λ). Any particular wavelength, (λ), of imaged light of the object isassociated with a set of spectral images each featuring spectralfingerprints of the object in two dimensions, for example, along the xand y directions, whereby voxels having that value of wavelengthconstitute the pixels of a monochromatic image of the object at thatwavelength. Each spectral image, featuring a range of wavelengths ofimaged light of the object is analyzed to produce a two dimensional mapof one or more physicochemical properties, for example, geometricalshape, form, or configuration, and dimensions, and/or chemicalcomposition, of the object and/or of components of the object, in ascene or sample.

Spectral profiles treat image cubes as a collection of spectra andprovide a detailed and comprehensive description of the image cube as awhole. They use a set of characteristic spectra and their relativeoccurrences within an image cube to summarize the material compositionof the sample. The number of characteristic spectra extracted willdepend upon the variability in the material, and normally range from afew to a few dozen. Spectral profiles are created by matching eachspectra in an image cube to a characteristic spectra. The number ofspectra matched to each characteristic spectra are counted andnormalized to create the spectral profile of an image cube. Thus,spectral profiles can be thought of as a fingerprint derived from thehyperspectral image cube of the tobacco sample.

In hyperspectral imaging, multiple images of each object are generatedfrom object-emitted electromagnetic radiation having wavelengths andfrequencies associated with different selected parts or bands of anentire spectrum emitted by the object. For example, hyperspectral imagesof an object are generated from object emitted electromagnetic radiationhaving wavelengths and frequencies associated with one or more of thefollowing bands of an entire spectrum emitted by the object: the visibleband, spanning the wavelength range of about 400-700 nanometers, theinfra-red band, spanning the wavelength range of about 700-3000nanometers, and the deep infra-red band, spanning the wavelength rangeof about 3-12 microns. If proper wavelengths and wavelength ranges areused during hyperspectral imaging, data and information of thehyperspectral images could be optimally used for detecting and analyzingby identifying, discriminating, classifying, and quantifying, the imagedobjects and/or materials, for example, by analyzing different signaturespectra present in pixels of the hyperspectral images.

A high speed hyperspectral imaging system is often required fordifferent types of repeatable and non-repeatable chemical and physicalprocesses taking place during the sub-100 millisecond time scale, whichcannot, therefore, be studied using regular hyperspectral imagingtechniques. Combustion reactions, impulse spectro-electrochemicalexperiments, and inelastic polymer deformations, are examples of suchprocesses. Remote sensing of objects in distant scenes from rapidlymoving platforms, for example, satellites and airplanes, is anotherexample of a quickly changing observable that is often impossible torepeat, and therefore requires high speed hyperspectral imaging.

Disclosed herein is a method for detecting foreign matter (including oilor lubricant in product) within an agricultural product stream, such astobacco, via hyperspectral imaging and analysis. In certain formsthereof, provided are methodologies, protocols, procedures and equipmentthat are highly accurate and highly precise, in that they arereproducible and robust, when evaluating agricultural products, such astobacco. The testing methodologies disclosed herein exhibit highsensitivity, high resolution, and high speed during automatic on-lineoperation.

Certain forms disclosed herein are specifically focused on the domainencompassing measuring, analyzing, and determining, micro scaleproperties, characteristics, features, and parameters of agriculturalproducts, such as tobacco, generally with respect to individual tobaccosamples, and specifically with respect to single or individual tobaccoleaves contained within the tobacco samples, and more specifically withrespect to a wide variety of numerous possible physical, chemical,and/or biological properties, characteristics, features, and parametersof single or individual tobacco leaves contained within a given tobaccobale, lot or sample. In one form, provided is a method for removingforeign matter from an agricultural product stream of a manufacturingprocess employing hyperspectral imaging and analysis.

Certain forms disclosed herein use what will be referred to as“hyperspectrally detectable and classifiable codes.” As used herein, a“hyperspectrally detectable and classifiable code” is a micro scaleproperty, characteristic, feature, or parameter of a particular bulkagricultural product, such as a tobacco sample, which is hyperspectrallydetectable by hyperspectral imaging and analysis in a manner that theresulting hyperspectral data and information, for example, hyperspectral“fingerprint” or “signature” patterns are usable for classifying atleast part of a single or individual tobacco leaf contained within thatparticular tobacco sample. In turn, the classified part of the single orindividual tobacco leaf contained within that particular tobacco sampleis usable as part of a procedure for monitoring tobacco processing andmay also be used to remove foreign matter from an agricultural productstream.

Accordingly, a “hyperspectrally detectable and classifiable code” isdefined, generally with respect to a particular individual agriculturalproduct, such as a tobacco sample, and specifically with respect to asingle or individual tobacco leaf contained within the particulartobacco sample, and more specifically with respect to a physical,chemical, and/or biological property, characteristic, feature, orparameter, of that single or individual tobacco leaf contained withinthat particular tobacco sample. The hyperspectrally detectable andclassifiable codes are usable as part of a procedure for monitoring theprocessing or manufacturing of an agricultural product, such as tobacco.

Primary examples of micro scale testing for generating hyperspectrallydetectable and classifiable codes, include: physical(geometrical/morphological) shape or form and size dimensions of singleor individual tobacco leaves; coloring of single or individual tobaccoleaves; moisture (water) content of, or within, single or individualtobacco leaves; type, distribution, and compositional make-up, of(organic and inorganic) chemical species or components, of single orindividual tobacco leaves; types, distribution, and compositionalmake-up, of possible unknown or foreign (physical, chemical, and/orbiological) matter or species and aspects thereof on, and/or within,single or individual tobacco leaves; activity and/or reactivity ofsingle or individual tobacco leaves in response to physical stimuli oreffects, such as exposure to electromagnetic radiation; activity and/orreactivity of single or individual tobacco leaves in response tochemical stimuli or effects, such as exposure to aqueous liquids or tonon-aqueous (organic based) liquids; and activity and/or reactivity ofsingle or individual tobacco leaves in response to biological stimuli oreffects, such as exposure to biological organisms; physical(geometrical/morphological) shape or form and size dimensions of singleor individual tobacco leaves; coloring of single or individual tobaccoleaves; moisture content of, or within, single or individual tobaccoleaves; types, distribution, and compositional make-up, of (organic andinorganic) chemical species or components, of single or individualtobacco leaves; types, distribution and compositional make-up ofpossible unknown or foreign (physical, chemical, and/or biological)matter or species and aspects thereof on, and/or within, single orindividual tobacco leaves; activity and/or reactivity of single orindividual tobacco leaves in response to physical stimuli or effects,such as exposure to electromagnetic radiation; activity and/orreactivity of single or individual tobacco leaves in response tochemical stimuli or effects (such as exposure to aqueous (water based)liquids or to non-aqueous (organic based) liquids); and activity and/orreactivity of single or individual tobacco leaves in response tobiological stimuli or effects, such as exposure to biological organisms.

Accordingly, provided is a method for removing foreign matter (includingoil or lubricant in product) from an agricultural product stream of amanufacturing process. The method includes conveying a product streampast an inspection station; scanning a region of the agriculturalproduct stream as it passes the inspection station using at least onelight source of a single or different wavelengths; generatinghyperspectral images from the scanned region; determining a spectralfingerprint for the agricultural product stream from the hyperspectralimages; comparing the spectral fingerprint obtained in step (c) to aspectral fingerprint database containing a plurality of fingerprintsusing a computer processor to determine whether foreign matter ispresent and, if present, generating a signal in response thereto; andremoving a portion of the conveyed product stream in response to thesignal.

The method is based upon obtaining hyperspectral signatures for theagricultural material being processed to minimize or eliminate the needfor human evaluation during processing. To accomplish this, first astandard database is created that includes hyperspectral signaturestaken at the location of an inspection station. The database so obtainedis used as a benchmark against which the process will be monitored.

The method may comprise scanning multiple regions along the sample ofagricultural product using at least one light source of a single ordifferent wavelengths; and generating hyperspectral images from themultiple regions. The method may further comprise determining a code.

The agricultural product may comprise tobacco. The tobacco may be in theform of a sample. At least one light source may be positioned tominimize the angle of incidence of each beam of light with the tobacco.

The method may further comprise repeating the steps of scanning at leastone region along a sample of agricultural product using at least onelight source of different wavelengths, generating hyperspectral imagesfrom the at least one region, and forming a spectral fingerprint for thesample of agricultural product from the hyperspectral images, for aplurality of samples of agricultural product during the processing ofthe agricultural product.

The method may further comprise storing spectral fingerprints data froma plurality of samples of agricultural product taken at various stagesof processing within computer storage means to form a process database.

The method may further comprise storing data about the spectralfingerprint within a computer storage means, and repeating the steps ofscanning multiple regions along a sample of a desirable agriculturalproduct using at least one light source of a single or differentwavelengths, generating hyperspectral images from the multiple regions,forming a spectral fingerprint for the sample from the hyperspectralimages, storing data about the spectral fingerprint within a computerstorage means, using a plurality of desirable agricultural products. Thedesirable agricultural product may be an unprocessed, semi-processed orfully processed agricultural product, such as tobacco.

While the invention is described in detail for the case of tobacco, itshould be understood that tobacco is used only to illustrate the methodsand systems contemplated herein and not so as to limit the applicationof the methods and systems described herein.

Referring now to FIG. 2, one form of a detection and separation system100, as disclosed herein, is shown schematically. In operation, anagricultural product stream 112, which may be a tobacco stream,containing foreign material, such as foil, cellophane, warehouse tags,and paper, or oil or lubricant containing material, is delivered from aprocessing line by conveyor 114. Conveyor 114 is preferably a vibratinginclined conveyor which vibrates as shown by arrows V. In one form,conveyor 114 ends above another conveyor 116, which can be an ordinaryconveyor belt, and is spaced vertically above conveyor 116 a sufficientdistance to accommodate the remainder of the system described below. Asproduct stream 112 reaches the end of conveyor 114, it drops under theinfluence of gravity in a cascade C to conveyor 116. In one form,because conveyor 114 is inclined, the product stream does not have asgreat a horizontal velocity when it falls, so that cascade C does nothave any significant front-to-back horizontal spread.

In another form, detection and separation system 100 may include asingle conveyor 114 for inspecting finished product, such as cigarettes,smokeless tobacco containers, SNUS pouches, etc., with productrejection, described in more detail below, taking place on the sameconveyor.

The detection and separation system 100 includes at least one lightsource 118 for providing a beam of light. As shown, the at least onelight source 118 may be mounted on an arm 144 for positioning at leastone light source 118 in proximity to the product stream 112. In oneform, arm 144 may be mounted to cabinet 146 and may be either fixedthereto or moveably positionable, as will be described hereinbelow. Asecond light source (not shown) may also be provided and mounted tocabinet 146 or, optionally, to another arm (not shown), which in turnmay be mounted to cabinet 146.

In some forms, the at least one light source 118 for providing a beam oflight of different wavelengths comprises a tungsten, halogen or a xenonlight source. In another form, the at least one light source 118 forproviding a beam of light comprises a mercury light source. In yetanother form, the at least one light source 118 or the second lightsource (not shown) comprises an ultraviolet light source for use inproviding a chemical signature of the agricultural product stream 112.This optional ultraviolet light source adds an additional media ofclassification that provides a better understanding of an agriculturalproduct's characteristics. In still yet another form, the at least onelight source 118 comprises a xenon light source, the second light source(not shown) comprises a mercury light source and a third light source(not shown) comprises an ultraviolet light source.

As shown in FIG. 2, the light from the at least one light source 118 maybe directed toward the cascade C of the product stream P₀ by the mirror120. The hyperspectral image of a scene or a sample is obtained usinghyperspectral imaging camera 128, which, in some forms, may receivelight reflected by mirror 130, as shown.

In some forms, the at least one light source 118 and/or the second lightsource (not shown) may be positioned to minimize the angle of incidenceof a beam of light with the agricultural product stream P₀.

In order to segregate ambient light from the light provided by system100, walls (not shown) may be added to form an enclosure to provide adark-room-like environment for system 100.

Referring still to FIG. 2, a computer 124 may be mounted with cabinet146. The computer 124 is provided with a processor capable of rapidlyhandling system data and programmed to compare the detected componentwavelengths to a database of previously analyzed agricultural products.Computer 124 may be a personal computer having an Intel® Core™2 Quad orother processor. Computer 124 may also control the operation of thesystem 100. A device for providing an uninterrupted source of power tocomputer 124 may be provided and mounted within cabinet 146, suchdevices readily available from a variety of commercial sources. Aregulated power supply (not shown) may be provided to assure that atightly controlled source of power is supplied to system 10.

In one form, system 100, is provided with a user interface 140 thatenables an operator (not shown) to observe and control variousoperational aspects of the system 100. The user interface 140 mayinclude a CRT or LCD panel for output display. For input, the userinterface 140 may include a keyboard, touch-screen or other input meansknown in the art. The operator can view representations of the articlesin the product stream 112 as they are processed in system 100 on theuser interface 140. Yet further, the user interface 140 provides a meansfor the operator to configure the operation of system 100 to make adetermination between acceptable product and undesirable product. Datagathered by the user interface 140 and provided to the user interfaceare transported as user interface data 142.

When system 100 detects foreign material in product stream 112, computer124 sends a signal to ejector manifold 132, which is positioned indownstream relation to the region illuminated by the at least one light118. Ejector manifold 132 is in fluid transmission relation to thetrajectory of the product stream 112. The ejector manifold 132 includesa plurality of ejector nozzles 134, which are individually directed andcontrolled to selectively remove undesirable product material 136 fromthe product stream 112. The ejector nozzles 134 act as conduits fordirecting fluid pulses to dislodge or otherwise re-direct productmaterial traveling in the trajectory. Individual ejector nozzles 134contained in the ejector manifold 132 are driven by a plurality ofremoval signals, which may be provided by processor 124.

Ejector nozzles 134 are connected to a source of high pressure fluidwhich is preferably air at approximately 80 psi, although other gases,such as steam, or liquids, such as water, can be used. When one ofejector nozzles 134 opens in response to a signal, a blast of air A isdirected against that portion of cascade C in which the foreign materialwas detected to force that portion 136 of the product stream and/orforeign material to fall into receptacle 138 for manual sorting, ifnecessary. In the case of usable product, it may be returned to theproduct processing line upstream or downstream of system 100, dependingon whether or not rescanning is desired. Alternatively, portion 136could be deflected to a conveyor that removes it to another area forprocessing.

As may be appreciated, system 100 allows tobacco or other materials tobe processed at greater rates than a system in which the tobacco orother materials are scanned on a belt conveyer. This is because whenproduct is optically scanned on a belt, it has to be in a “monolayer,”or single layer of particles, for all of the particles on the belt to bevisible to the hyperspectral imaging camera 128. However, as the tobaccoor other material falls in cascade C, relative vertical motion betweenthe various particles of tobacco and foreign material is induced by theturbulence of the falling stream, so there is a greater probability thata particular piece of foreign material will be visible to hyperspectralimaging camera 128 at some point in its fall. Relative vertical motionalso results if the foreign material is significantly lighter or heavierthan tobacco so that it has greater or less air resistance as it falls.Relative vertical motion is enhanced by the vibration of conveyor 114which brings lighter material to the surface of the tobacco before itfalls in cascade C, making the lighter material, which is usuallyforeign material, easier to detect, as in a monolayer.

The inclination of conveyor 114, in reducing the horizontal spread ofcascade C as discussed above, also enhances relative vertical motionbecause the particles in cascade C have little or no horizontal velocitycomponent. Any horizontal velocity component that a particle has when itfalls off conveyor 114 is small because conveyor 114 is inclined, andair resistance quickly reduces the horizontal motion to near zero. Therelative vertical motion allows a relatively thicker layer of tobacco orother material to be scanned, so that a greater volume can be scannedper unit of scanning area. Given a constant rate of area scanned perunit time, the increased volume scanned per unit area translates into ahigher volume of tobacco or other material scanned per unit time.

Results obtained by system 100 are based on the scanning and counting ofindividual samples, each comprising dozens of scans, and each sampleclassified using spectral band features, spectral finger prints (SFP),major spectral representative components, purity and quality of eachmajor compound (component, SFP), relative quantity of each SFP and,optionally, crystallization and morphological features.

In operation, a plurality of samples of agricultural products, such astobacco samples are scanned, which relate to the location in themanufacturing process where an inspection station employing system 100will be installed. As indicated, for agricultural products, such astobacco, a significant number of samples should be scanned in order thatthe impact of sample variability is reduced. In practice, it has beenobserved that the impact due to this variability can be reduced when thenumber of samples N is about 5 to about 25. However, by carefullyselecting representative samples, fewer samples could be used toincorporate all the normal variations observed in processing aparticular product. Applying this technique to tobacco, tobacco samplesmay be scanned using xenon and/or mercury and/or tungsten, and/orhalogen light sources and an optional ultraviolet light source may beused for chemical signature classification.

In operation, the light source(s) is activated (one, two, three or morespot lights in parallel to the region of interest (ROI)), permittingredundant data to be gathered. A plurality of regions of interest (suchas by way of example, but not of limitation, a 20 cm×20 cm area for eachROI) is scanned for each sample to provide one, two, three or morehyperspectral images. Scanning is performed and the reflection spectralsignature and optional fluorescence spectral chemical signaturereceived. The images are then saved and a database (including labelsidentifying the particular sample and/or lot) is thus formed from thecombined information obtained for the N samples

During scanning, the hyperspectral camera system provides a threedimensional hyperspectral image cube. The image cube, which may be, byway of example, but not of limitation, on the order of about a 696 pixelby 520 pixel array. Such a picture or frame would thus contain 361,920pixels. As may be appreciated by those skilled in the art, each pixelmay contain about 128, 256, 500 or more spectra points at differentwavelengths for an agricultural product such as tobacco.

Referring now to FIG. 3, an algorithm 200 for use in the system andmethods described herein will now be disclosed. In step 210, a darkimage and reference image are obtained for use in system calibration. Instep 220, the reference image is analyzed and calibration coefficientsare obtained. In step 230, a hyperspectral image of a tobacco sample isobtained. In step 240, using the information obtained duringcalibration, dark values are removed and the sample image normalized. Instep 250, calibration coefficients are applied to compensate forfluctuations in operating conditions (e.g., light intensity, ambientconditions, etc.). In step 260, steps 230-250 are repeated for allsamples and the data so obtained is added in step 270 to the database ofspectral hypercubes (whole dataset). It should be understood that thealgorithm for creating the database as illustrated in FIG. 3 isreferenced to tobacco samples by way of illustration only and not aslimiting. The same steps could be used to create the whole spectraldatabase for application in other agricultural products like tea, fruitsgrapes or other products. The result is 350, a spectral library for allthe samples that could be used to assess and monitor processing of theagricultural product, which contains the spectral fingerprints of thedataset found in step 340, and the unique spectra found in step 330.

Referring now to FIG. 4, a method for analyzing data to create aspectral library of samples 300, in accordance herewith is shown.Spectral library 300 is formed by obtaining a dataset in step 310. Instep 320, the dataset is preprocessed. Unique spectra, indicative of thedataset, are identified in step 330. In step 340, spectral distributions(spectral fingerprints) are found for each of the samples using theunique spectra found in step 330.

In some forms, following imaging, several data processing routines areperformed to reduce noise, increase consistency, enhance spectra forfeature extraction, and reduce computation time. The data processingroutines are summarized below.

Spectral binning is used to produce more consistent signals, reduce filesize, and decrease processing time. Spectral binning is an operationperformed on each spectra of the image cube, and consists of summationof adjacent wavelengths to produce a down-sampled spectra. Down-sampledspectra have less resolution, but increased signal to noise ratio. Asampling rate was chosen that creates a signal with maximum compressionand minimal loss of fidelity. This is followed by median filtering toreduce the noise and increase the signal to noise ratio.

Spatial Binning is applied to the image cube, and consists ofdown-sampling via summation of adjacent pixels. Pixels contain spectra,so spatial binning results in summation of adjacent spectra. Thisincreases signal to noise ratio, reduces file size, and decreasesprocessing time. Information loss is minimal because the camera isselected to provide a high resolution. Summation allows for each pixel'sspectra to contribute to the down-sampled spectra, and adjacent pixelsare usually from the same part of the leaf, which tend to have similarspectra. Similar effects may be achieved by using a Gaussian filter toincrease spatial coherency and to reduce “salt and pepper” pixelclassification. A Gaussian filter may be useful when spatial reductionis not desirable.

In some forms, image correction may be performed by first collectingdark images twice daily. These are then used during processing toestimate and remove sensor noise. The dark image processing procedureconsists of two steps. Dark images' spectra, {right arrow over( )}d_(j)∈D_(i) are first binned to be consistent with other imagescubes. Then the spectral mean, {right arrow over ( )}d^(mean;i) iscalculated (Eq. 1) and used to estimate sensor noise.

$\begin{matrix}{{\overset{arrow}{s}}_{i}^{mean} = {{\sum\limits^{\forall{{\overset{arrow}{s}}_{j} \in S_{i}}}\;{\overset{arrow}{s}}_{j}}}} & (1)\end{matrix}$

During data processing {right arrow over ( )}d^(mean;)i is removed fromeach spectra of each sample by subtraction. This is a standard method ofremoving sensor noise.

Reference image cubes, R_(i) are collected twice-daily and containspectra that can be used to measure and correct lightinginconsistencies. Applying a correction based on the reference image willalso eliminate any sensor drift or variation over time. This step isvery important as any changes in either the lighting conditions or thesensor response drift will adversely affect the system performance.Also, pixels of shadow have low signal to noise ratio and should beeliminated.

Most signals tend to have a maximum peak created by the spectralsignature of the light whose value is proportional to the amount oflight hitting the sensor, and the sensor's response. For this reasonshadow detection is applied using maximum peak threshold ing.

${pixel} = \{ \begin{matrix}{{not}\mspace{14mu}{shadow}\text{:}} & {{\max( {\overset{arrow}{s}}_{i} )} \geq {thresh}} \\{{shadow}\text{:}} & {{\max( {\overset{arrow}{s}}_{i} )} < {thresh}}\end{matrix} $where max({right arrow over ( )}si) returns the maximum component of thespectra, {right arrow over ( )}si.

Spectra with a max peak less than a user-defined threshold are tagged asshadow spectra, and ignored during spatial binning, spectral binning,local spectra extraction, and spectra matching.

Image cubes of tobacco samples contain thousands of spectra, many ofwhich are nearly identical and correspond to similar material propertiesand sensorial effects. The information contained in the image cube canbe summarized as a spectral profile using a set of characteristicspectra and their occurrence rate within a sample. Construction of aspectral profile consists of two primary steps.

1. Spectra Extraction—finding characteristic spectra.

2. Spectra Matching—matching an image cube's spectra to characteristicspectra.

The first step of building a spectral profile is to create a set ofcharacteristic spectra often called end-members. End-members are oftenmanually selected by choosing pixels that are known to correspond to aspecific class or contain a unique material. For many agriculturalproducts, including tobacco, distinct characteristics can be verydifficult to detect manually and class specific spectra arenon-existent. In such cases an automated spectra extraction technique ispreferable.

Spectra extraction differs from other automated end-member extractiontechniques in that it divides the spectral feature space using an evenlyspaced grid. Spectra extraction finds all spectra in a data set that aremore dissimilar than a user-defined threshold, α*. The assumption isthat if two spectra are more similar than α* they represent identicalmaterials and can be considered duplicates. By finding all dissimilarspectra in a data set, all unique materials can be found. Otherautomated spectra extraction algorithms are often associated withun-mixing models, which assume individual pixels contain a combinationof unique spectral signatures from multiple materials. SequentialMaximum Angle Convex Cone (SMACC) and Support Vector Machine-BasedEnd-member extraction are two examples. The technique disclosed hereinwas chosen for simplicity. Un-mixing models may not be appropriate forsmaller scale agricultural imaging, where individual pixel size is inmillimeters as opposed to aerial imaging, where individual pixel size isusually in meters. The spectra extraction procedure disclosed hereinfirst analyzes image cubes independently in a step called local spectraextraction. The results are then combined during global spectraextraction. Extraction in this order decreases processing time andallows for outliers of individual image cubes to be eliminated.

Once the characteristic spectra of the data set have been extracted,spectra matching step is applied on each image cube. Each {right arrowover ( )}s_(j)∈S_(i) is matched with the most similar {right arrow over( )}c_(k)∈C_(all). As an image cube is analyzed, a tally of the numberof matches for each {right arrow over ( )}c_(k)∈C_(all) is kept as aspectral profile, {right arrow over ( )}p_(i). Each component ofcorresponds to the number of matches for a single {right arrow over( )}c_(k)∈C_(all). Once all {right arrow over ( )}s_(j)∈S_(i) have beenmatched {right arrow over ( )}p_(i) is normalized and represents thepercent occurrence rate of each {right arrow over ( )}c_(k)∈C_(all) inSi.

The inclusion of unidentified pixels can be advantageous for certainscenarios such as when tobacco samples contain non-tobacco material, ifshadow detection is unreliable, or if only a selected few spectra shouldbe included in the spectral profiles. Rather than forcing a match withthe most similar {right arrow over ( )}c_(k)∈C_(all), Spectra that aremore dissimilar than α** to all {right arrow over ( )}c_(k)∈C_(all) arecounted as unidentified. Unidentified pixels do not contribute to thespectral profile. α** is a user-defined parameter, and larger values ofα** will allow more dissimilar spectra to match to {right arrow over( )}c_(k)∈C_(all), while small values will allow matches with onlysimilar spectra. Setting a high α* will force a match with the closest{right arrow over ( )}c_(k)∈C_(all).

Since previous hyperspectral image classification problems have focusedon pixel by pixel classification, spectra matching is often the goal ofhyperspectral image analysis. These applications often make use ofmachine learning algorithms such as support vector machines and decisiontrees to classify spectra. Matching in this manner requires a trainingset of characteristic spectra which is not practical. Fully automatedtechniques are preferable, and also may use spectral feature fitting(SFF). SFF is designed to distinguish between spectra using specificfeatures of a spectra. While SFF can achieve successful results,spectral angle measure (SAM) is a more appropriate measure, since it isintended to find the similarity between two spectra using all the bandsas confirmed by the results.

The goal of feature selection is to choose a subset of features capableof summarizing the data with little or no information loss. It isapplied before classification to avoid dimensionality, which oftenreduces classification performance. Spectral profiles can containredundant features, particularly when the spectra extraction threshold,α* is low. Experimental work suggests that selection of an appropriateα*combined with the ability of support vector machines (SVM) to handleredundant and/or uninformative features eliminates the need for afeature selection step. However in some cases we found that choosing theoptimal subset of features using the Jeffreys-Matusita Distance as aninformation measure was effective.

In some forms, classification may be conducted using well-knownprocedures, by way of example and not of limitation, discriminantanalysis, SVM, neural networks, etc., as those skilled in the art willrecognize.

As may be appreciated, the amount of raw data is very large, due to thelarge number of pixels existing within a particular image cube. Aseparate spectral is obtained for each pixel. Advantageously, the methoddisclosed herein identifies a set of characteristic spectra for thewhole dataset. The composition of the spectra provides a signature forthe sample, with similar samples having similar signatures orfingerprints. Unique spectral fingerprints are then identified for eachtobacco sample.

The spectral library or database of the samples developed in FIG. 4 maybe used to determine whether foreign matter is present in anagricultural product stream. This information may also be used in aclosed system capable of separating foreign matter from the agriculturalproduct stream, as disclosed herein.

Advantageously, the method and system disclosed herein provides achemical imaging platform that enables speed and exceptionally highsensitivity, thus accurate and non-destructive nature, whereinmeasurement time is greatly reduced. This enables the tracking of themonitored material with high repeatability.

The method and system disclosed herein provides a highly sensitive hyperspectral imaging analyzer with co-sharing database capabilities. Thedigitized highly sensitive imaging system disclosed herein enables theimaging of materials components, more detailed observation and providesmore specific, sensitive, accurate and repeatable measurements. This maybe achieved using advanced image recognition technologies, as well asadaptive and collaborative data bases and statistical and optimizationalgorithms.

The method and system disclosed herein is capable of characterizing thecomposition of inorganic, organic, and chemical particulate mattersuspended in agricultural products such as tobacco. The instrument scansleaf or other samples, analyses the scanned samples' wavelengthsignature, performing trends analysis and compares the gathered data todata bases that may, in one form, be continuously updated. The systemthen generates reports for system users. A remote network may beprovided to support the gathering of data for integrated databaseconstruction as well as to support remote professional personnel in realtime.

Advantageously, the proposed method requires no sample preparation. Inoperation, linear calibration plots in the ppm range are obtained forthe presence of mono-components and for simple compound mixtures in thismatrix. Non-contact, non-destructive, near real time on-line, automatedphysicochemical imaging, classification and analysis of a sample oftobacco leaves or other agricultural products is provided without theneed for consumable materials for sample processing. The system operatesusing algorithms and software packages, together with unique ultra-highresolution optical components and a two-dimensional sample positioningof regions of interest for generating, for example, five dimensional(5D) spectral images of the tobacco sample under analysis.

All or a portion of the devices and subsystems of the exemplary formscan be conveniently implemented using one or more general purposecomputer systems, microprocessors, digital signal processors,microcontrollers, and the like, programmed according to the teachings ofthe exemplary forms disclosed herein, as will be appreciated by thoseskilled in the computer and software arts.

In view thereof, in one form there is provided a computer programproduct for monitoring a manufacturing process of an agriculturalproduct, the computer program structured and arranged to determinewhether foreign matter is present in an agricultural product stream,including one or more computer readable instructions embedded on atangible computer readable medium and configured to cause one or morecomputer processors to perform the steps described above andtransmitting information relating to the steps described above over acommunications link.

Appropriate software can be readily prepared by programmers of ordinaryskill based on the teachings of the exemplary forms, as will beappreciated by those skilled in the software art. Further, the devicesand subsystems of the exemplary forms can be implemented on the WorldWide Web. In addition, the devices and subsystems of the exemplary formscan be implemented by the preparation of application-specific integratedcircuits or by interconnecting an appropriate network of conventionalcomponent circuits, as will be appreciated by those skilled in theelectrical art(s). Thus, the exemplary forms are not limited to anyspecific combination of hardware circuitry and/or software.

Stored on any one or on a combination of computer readable media, theexemplary forms disclosed herein can include software for controllingthe devices and subsystems of the exemplary forms, for driving thedevices and subsystems of the exemplary forms, for enabling the devicesand subsystems of the exemplary forms to interact with a human user, andthe like. Such software can include, but is not limited to, devicedrivers, firmware, operating systems, development tools, applicationssoftware, and the like. Such computer readable media further can includethe computer program product of a form disclosed herein for performingall or a portion (if processing is distributed) of the processingperformed in implementing the methods disclosed herein. Computer codedevices of the exemplary forms disclosed herein can include any suitableinterpretable or executable code mechanism, including but not limited toscripts, interpretable programs, dynamic link libraries (DLLs), Javaclasses and applets, complete executable programs, Common Object RequestBroker Architecture (CORBA) objects, and the like. Moreover, parts ofthe processing of the exemplary forms disclosed herein can bedistributed for better performance, reliability, cost, and the like.

As stated above, the devices and subsystems of the exemplary forms caninclude computer readable medium or memories for holding instructionsprogrammed according to the forms disclosed herein and for holding datastructures, tables, records, and/or other data described herein.Computer readable medium can include any suitable medium thatparticipates in providing instructions to a processor for execution.Such a medium can take many forms, including but not limited to,non-volatile media, volatile media, transmission media, and the like.Non-volatile media can include, for example, optical or magnetic disks,magneto-optical disks, and the like. Volatile media can include dynamicmemories, and the like. Transmission media can include coaxial cables,copper wire, fiber optics, and the like. Transmission media also cantake the form of acoustic, optical, electromagnetic waves, and the like,such as those generated during radio frequency (RF) communications,infrared (IR) data communications, and the like. Common forms ofcomputer-readable media can include, for example, a floppy disk, aflexible disk, hard disk, magnetic tape, any other suitable magneticmedium, a CD-ROM, CDRW, DVD, any other suitable optical medium, punchcards, paper tape, optical mark sheets, any other suitable physicalmedium with patterns of holes or other optically recognizable indicia, aRAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip orcartridge, a carrier wave or any other suitable medium from which acomputer can read.

The forms disclosed herein, as illustratively described and exemplifiedhereinabove, have several beneficial and advantageous aspects,characteristics, and features. The forms disclosed herein successfullyaddress and overcome shortcomings and limitations, and widen the scope,of currently known teachings with respect to the processing ofagricultural products such as tobacco.

The forms disclosed herein, provide for processing methodologies(protocols, procedures, equipment) for, by way of example, but not oflimitation, tobacco samples, which are highly accurate and highlyprecise (reproducible, robust). The forms disclosed herein, provide highsensitivity, high resolution, and high speed (fast, at short timescales), during automatic operation, in an optimum and highly efficient(cost effective) commercially applicable manner.

As may be appreciated, the performance of the methods and systemsdisclosed herein is dependent of the number of regions and samplesscanned, image sizes, light sources, filters, light source energystability, etc.

EXAMPLES Example 1

The operation of the system and a method of forming a database will nowbe described.

The system was initiated and the light sources brought up to operatingtemperature. A dark image and reference image was obtained for systemcalibration. The reference image was analyzed and calibrationcoefficients obtained. A hyperspectral image of a tobacco sample wasobtained.

Using the information obtained during the aforementioned calibration,dark values were removed and the sample image normalized. Calibrationcoefficients were applied to compensate for fluctuations in operatingconditions (e.g., light intensity, ambient conditions, etc.).Hyperspectral images for additional tobacco samples, and NTRM sampleswere obtained for additional samples of interest and all data soobtained is added to the database of spectral hypercubes (wholedataset).

A spectral library was formed from the database of spectral hypercubesby first preprocessing the data. Unique spectra, indicative of thedataset were identified and samples mapped to the spectral fingerprintsso obtained. The unique spectra were then added to a spectral database.

From the data so obtained, a graph of intensity as a function ofwavelength for tobacco and non-tobacco related material is presented inFIG. 5. As may be seen, sharp distinctions were observed for tobacco andpolymeric and metallic NTRM.

Example 2

A method for using the database created, in accordance herewith, willnow be described by way of this prophetic example.

When a new batch of agricultural product is to be processed, first, aspectral distribution is obtained for this product based on theteachings of this invention. As described herein, a selection algorithmis used, the results of which provide the parameters necessary todetermine whether foreign matter is present in an agricultural productstream. Such material may be isolated and removed from the productstream in response to the determination.

Upon implementation in accordance with the foregoing teachings, thesystem will lessen or obviate the need for off-line sampling in thequality control of commercially manufactured agricultural products.

As used herein the terms “adapted” and “configured” mean that theelement, component, or other subject matter is designed and/or intendedto perform a given function. Thus, the use of the terms “adapted” and“configured” should not be construed to mean that a given element,component, or other subject matter is simply “capable of” performing agiven function but that the element, component, and/or other subjectmatter is specifically selected, created, implemented, utilized,programmed, and/or designed for the purpose of performing the function.It is also within the scope of the present disclosure that elements,components, and/or other recited subject matter that is recited as beingadapted to perform a particular function may additionally oralternatively be described as being configured to perform that function,and vice versa.

Illustrative, non-exclusive examples of systems and methods according tothe present disclosure are presented in the following enumeratedparagraphs. It is within the scope of the present disclosure that anindividual step of a method recited herein, including in the followingenumerated paragraphs, may additionally or alternatively be referred toas a “step for” performing the recited action.

A1 A method for removing foreign matter from an agricultural productstream of a manufacturing process, the method comprising the steps of:(a) conveying a product stream past an inspection station; (b) scanninga region of the agricultural product stream as it passes the inspectionstation using at least one light source of a single or differentwavelengths; (c) generating hyperspectral images from the scannedregion; (d) determining a spectral fingerprint for the agriculturalproduct stream from the hyperspectral images; (e) comparing the spectralfingerprint obtained in step (c) to a spectral fingerprint databasecontaining a plurality of fingerprints using a computer processor todetermine whether foreign matter is present and, if present, generatinga signal in response thereto; and (f) removing a portion of the conveyedproduct stream in response to the signal.

A2. The method of paragraph A1, further comprising the step of causingthe portion of the conveyed product stream to fall under the influenceof gravity in a cascade.

A3. The method of paragraph A2, wherein the cascade is a turbulentcascade.

A4. The method of paragraph A2, wherein said step of removing a portionof the conveyed agricultural product stream in response to the signalfurther includes directing fluid under pressure at the portion of theagricultural product stream.

A5. The method of paragraph A4, wherein the fluid is a gas.

A6. The method of paragraph A5, wherein the gas is pressurized air.

A7. The method of paragraph A1, wherein the agricultural product istobacco.

A8. The method of paragraph A1, wherein the agricultural product is tea.

A9. The method of paragraph A1, wherein the at least one light source ispositioned to minimize the angle of incidence of each beam of light withthe agricultural product stream.

A10. The method of paragraph A1, wherein the at least one light sourcefor providing a beam of light comprises a light source selected from thegroup consisting of a tungsten light source, a halogen light source, axenon light source, a mercury light source, an ultraviolet light source,and combinations thereof.

B1. A system for detecting foreign matter within a agricultural productstream, comprising: (a) a first conveying means for delivering a productstream; (b) an inspection station comprising (i) at least one lightsource of a single or different wavelengths for providing a beam oflight to scan a region of the agricultural product stream as it passesthe inspection station, and (ii) a hyperspectral camera system forproviding a three dimensional hyperspectral image cube; (c) a computerprocessor structured and arranged to determine a spectral fingerprintfor the agricultural product stream from the hyperspectral image cubeand to compare the spectral fingerprint obtained to a spectralfingerprint database containing a plurality of fingerprints to determinewhether foreign matter is present and, if present, generating a signalin response thereto.

B2. The system of paragraph B1, further comprising at least onedeflecting system responsive to the signals obtained from said computerprocessor, said at least one deflecting system directing fluid underpressure at a portion of the product stream when said computer processordetermines that foreign matter is present in the product stream.

B3. The system of paragraph B2, wherein the fluid so directed iseffective to remove the foreign matter.

B4. The system of paragraph B3, further comprising a second conveyingmeans located below and spaced vertically from said first conveyingmeans for further conveying the product stream from said first conveyingmeans, wherein said product stream is transferred from said firstconveying means to said second conveying means by falling therebetweenunder the influence of gravity in a cascade.

B5. The system of paragraph B4, wherein the cascade is a turbulentcascade.

B6. The system of paragraph B1, wherein said first conveying means is aninclined vibrating conveyor.

B7. The system of paragraph B1, wherein the fluid is a gas.

B8. The system of paragraph B7, wherein the gas is air.

B9. The system of paragraph B1, wherein the agricultural product istobacco.

B10. The system of paragraph B1, wherein the agricultural product istea.

B11. The system of paragraph B1, wherein the at least one light sourceis positioned to minimize the angle of incidence of each beam of lightwith the agricultural product stream.

B12. The system of paragraph B1, wherein the at least one light sourcefor providing a beam of light comprises a light source selected from thegroup consisting of a tungsten light source, a halogen light source, axenon light source, a mercury light source, an ultraviolet light source,and combinations thereof.

C1. A method of creating a database for use in identifying foreignmaterial agricultural product that may be present in a manufacturingprocess for producing an agricultural product, the method utilizinghyperspectral imaging and comprising the steps of: (a) obtaining a darkimage and a reference image for calibration; (b) analyzing the referenceimage to obtain calibration coefficients; (c) obtaining a hyperspectralimage for an agricultural sample; (d) removing dark values andnormalizing the agricultural sample image; (e) applying calibrationcoefficients to compensate for fluctuations in system operatingconditions; (f) repeating steps (c)-(e) for all agricultural samples;(g) obtaining a hyperspectral image for a foreign material sample; (h)removing dark values and normalizing the agricultural sample image; (i)applying calibration coefficients to compensate for fluctuations insystem operating conditions; (j) repeating steps (g)-(i) for all samplesand (k) storing all hyperspectral sample hypercubes to form thedatabase.

C2. A computer database stored in a computer readable medium, producedin accordance with the method of paragraph C1.

It is to be fully understood that certain aspects, characteristics, andfeatures, of the forms disclosed herein, which are illustrativelydescribed and presented in the context or format of a plurality ofseparate forms, may also be illustratively described and presented inany suitable combination or sub-combination in the context or format ofa single form. Conversely, various aspects, characteristics, andfeatures, of the forms disclosed herein, which are illustrativelydescribed and presented in combination or sub-combination in the contextor format of a single form, may also be illustratively described andpresented in the context or format of a plurality of separate forms.

All patents, patent applications, and publications, cited or referred toin this specification are herein incorporated in their entirety byreference into the specification, to the same extent as if eachindividual patent, patent application, or publication, was specificallyand individually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisspecification shall not be construed or understood as an admission thatsuch reference represents or corresponds to prior art. To the extentthat section headings are used, they should not be construed asnecessarily limiting.

While the forms disclosed herein have been described in connection witha number of exemplary forms, and implementations, the forms disclosedherein are not so limited, but rather cover various modifications, andequivalent arrangements, which fall within the purview of the presentclaims.

What is claimed:
 1. A method for detecting foreign matter from anagricultural product stream of a manufacturing process, the methodcomprising: conveying a product stream past an inspection station;scanning a region of the conveyed product stream as the region of theconveyed product stream passes the inspection station using at least onelight source; generating one or more hyperspectral images in real timefrom the scanned region; determining a product fingerprint from the oneor more hyperspectral images in real time; comparing the productfingerprint in real time to a hyperspectral fingerprint databasecontaining a plurality of hyperspectral fingerprints using a computerprocessor; and determining whether foreign matter is present based onthe comparing the product fingerprint in real time to the hyperspectralfingerprint database.
 2. The method of claim 1, further comprising:causing a portion of the conveyed product stream to fall under theinfluence of gravity in a cascade.
 3. The method of claim 2, wherein thecascade is a turbulent cascade.
 4. The method of claim 1, furthercomprising: removing a portion of the conveyed product stream inresponse to a determination that foreign matter is present.
 5. Themethod of claim 4, wherein the removing the portion of the conveyedproduct stream includes directing fluid under pressure at the portion ofthe conveyed product stream.
 6. The method of claim 5, wherein the fluidis a gas.
 7. The method of claim 6, wherein the gas is pressurized air.8. The method of claim 1, wherein the agricultural product is tobacco.9. The method of claim 1, wherein the agricultural product is tea. 10.The method of claim 1, wherein the at least one light source ispositioned to reduce an angle of incidence of each beam of light withthe conveyed product stream.
 11. The method of claim 1, wherein the atleast one light source includes a tungsten light source, a halogen lightsource, a xenon light source, a mercury light source, an ultravioletlight source, or any combination thereof.
 12. The method of claim 1,wherein the foreign matter is an oil or a lubricant.
 13. A system fordetecting foreign matter within an agricultural product stream,comprising: an inspection station including, at least one light sourceconfigured to provide a beam of light to scan a region of a productstream as it passes the inspection station, and a hyperspectral camerasystem configured to provide a three dimensional hyperspectral imagecube from the scanned region in real time; and a computer processorconfigured to determine a product fingerprint from the hyperspectralimage cube and to compare in real time the product fingerprint to ahyperspectral fingerprint database containing a plurality ofhyperspectral fingerprints to determine whether foreign matter ispresent.
 14. The system of claim 13, further comprising: at least onedeflecting system responsive to signals obtained from said computerprocessor, the at least one deflecting system configured to remove aportion of the product stream in response to a determination of thecomputer processor that foreign matter is present in the product stream.15. The system of claim 13, further comprising: at least one deflectingsystem responsive to signals obtained from the computer processor, theat least one deflecting system configured to direct fluid under pressureat a portion of the product stream in response to a determination of thecomputer processor that foreign matter is present in the product stream.16. The system of claim 15, wherein the fluid so directed is effectiveto remove the foreign matter.
 17. The system of claim 15, wherein thefluid is a gas.
 18. The system of claim 17, wherein the gas is air. 19.The system of claim 13, wherein the agricultural product is tobacco. 20.The system of claim 13, wherein the agricultural product is tea.
 21. Thesystem of claim 13, wherein the at least one light source is positionedto reduce an angle of incidence of each beam of light with the conveyedproduct stream.
 22. The system of claim 13, wherein the at least onelight source includes a tungsten light source, a halogen light source, axenon light source, a mercury light source, an ultraviolet light source,or any combination thereof.
 23. The system of claim 13, wherein theforeign matter is an oil or a lubricant.